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ABSTRACT 


Artificial neural networks are based on mathematical models of biological networks, but it is not clear how similar these two networks are. 
We have recently demonstrated that we can mechanically manipulate single neurons and create functioning synapses. Here, we build on 
this discovery and investigate the feasibility and time scales to build an artificial neural network with biological neurons. To achieve this, 
we characterized the dynamics and forces when pulling functional axonal neurites using a micromanipulation technique with maximum 
speeds about 300 times faster than the average natural growth rate of 0.0017um/s. We find that the maximum force required to initiate and 
extend the neurites is about InN. The dynamics of the mechanical extension of the neurite is well described by many elastic springs and 
viscous dashpots in series. Interestingly, we find that the transport networks, specifically the actin network, lags behind the mechanically 
pulled structure. These insights could potentially open a new avenue to facilitate and encourage neuronal regrowth not relying on chemical 
queues. The extracted mechanical parameters and timescales characterize the neurite growth. We predict that it should be possible to use a 
magnetic trap to wire an artificial network such as a multi-layer perceptron in 17 hours. Once wired, we believe the biological neural network 
could be trained to process a hand-written digit using artificial neural network concepts applied to biological systems. We show how one 
could test the stability and robustness of this network by axotomizing (i.e. cutting) specific axons and reconnecting them using mechanical 
manipulation. 


© 2019 Author(s). All article content, except where otherwise noted, is licensed under a Creative Commons Attribution (CC BY) license 
(http://creativecommons.org/licenses/by/4.0/). https://doi.org/10.1063/1.5086873 


1. INTRODUCTION 


Inspired by R. Feynman’s statement ‘What I cannot create, I 
do not understand’ (written on his blackboard at the time of death 
in February 1988), we are investigating new methods to build neu- 
ronal networks using live neurons. Artificial neural networks seem 
to learn in a phenomenologically similar way to the brain. The brain 
is extensively interconnected, estimated to have about 86 billion neu- 
rons’ and around 100 trillion connections, it has feedback and feed- 
forward loops and the strength of its signal transmissions can be 
adjusted by adjusting the strength of its synapses.” Similarly, arti- 
ficial neural networks, which are based on mathematical models of 
a biological neural network, are highly interconnected, often incor- 
porate feedback and feedforward loops and the strength of certain 
connections can be adjusted through backpropagation learning.’ 
Despite these similarities, the two neural networks do not process 
information or learn in the same way. They also differ markedly in 
terms of power consumption. It is not currently possible to record 
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all the action potentials of a biological system. For example, every 
neuron and every connection in the model organism C. elegans 
have been mapped, but we cannot record from every neuron in that 
system.” 

In the present work, we further develop a micromanipula- 
tion technique that can be used to extend neurites from in-vitro 
axons to build a biological neural network where the topology of 
the network is exactly known and controlled, and all action poten- 
tials can in principle be measured. To achieve this, the neurons 
would be grown on a high-density multi-electrode array’ such that 
each neuron is on an electrode, while the axons would be mechan- 
ically manipulated to connect neurons. A simple artificial neural 
network, such as a multi-layer perceptron which could recognize 
hand-written digits,” could be built out of neurons, and learning 
algorithms could be tested by stimulating neurons using the multi- 
electrode array. In this way, this biological neural network could 
be directly compared to a topologically identical artificial neural 
network. 
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Growing an interconnected biological neural network with 
known topology is difficult due to the limited control and speed 
of axonal growth: for rat hippocampal neurons grown in-vitro, 
the average elongation rate is only 0.0017um/s.? By adhering to 
the axon and applying mechanical tension, Bray showed that neu- 
rites could be initiated out of the nucleus de novo and extended 
faster than normal growth.” Numerous subsequent experiments 
by Heidemann’s group showed that neurites can be character- 
ized as viscoelastic materials exhibiting growth at speeds which 
are directly proportional to the applied force. "° However, when 
a neurite is extended at speeds above 0.055um/s, the neurite 
thinned and broke.“ Growth in the central nervous system occurs 
through stationary periods and periods where the axonal elon- 
gation rate can be 1-2 orders of magnitude higher than the 
average rate,™ so an induced growth rate of 0.055um/s is not 
unreasonable. 

In previous experiments, we showed that by coating microbeads 
with a positively charged polymer (Poly-D-lysine (PDL) and netrin), 
an axon will form a synapse with the bead.''” By initiating the neu- 
rite from this synapse, we are able to pull at speeds of greater than 
0.33um/s over mm scale distances! '*’'” Twelve hours after being con- 
nected to another neuron, the neurites were shown to be function- 
ally connected using a double patch clamp measurement." Here, 
we build on these results by assuming neurites pulled by the same 
method would also be functional. We characterize the remarkable 
growth of these neurites and describe the growth using a model 
which consists of many springs and dashpots in series. We inves- 
tigate the limits and versatility of this biophysical system with the 
goal of determining the parameter space for building a perceptron 
from real neurons. We propose a method for building a multi-layer 
perceptron with 496 neurons and 4215 connections with the goal 
of eventually building a more complex system such as C. Elegans 
with 320 neurons, 6393 chemical synapses, 890 gap junctions and 
1410 neuromuscular junctions.’ Simpler biological circuits such as 
feedback or feedforward loops which require only a few neurons 
and electronic networks such as a 4-2-4 encoder and decoder could 
be similarly built using our technique to investigate the role of the 
diameter of the connecting axon on the transfer function of the 
neuronal system. 


Il. MATERIALS AND METHODS 
A. Neuronal cultures 


All neuronal cultures were approved by McGill University’s 
Animal Care Committee (Protocol #: 2013-7422) and conformed 
with the Canadian Council of Animal Care Guidelines. Hippocam- 
pal neurons were dissociated as described by Lucido et al., ° 
from Sprague Dawley rat embryos of either sex and plated 
on 100ug/mL Poly-D-Lysine (PDL) (Sigma-Aldrich) coated Mat- 
Tex dishes or Warner Instruments glass coverslips with neuron- 
specific microfluidic chambers designed by ANANDA Devices 
(https://anandadevices.com/) to ensure experiments were per- 
formed on axons. Cells were cultured for 7-21 days in-vitro 
(DIV), replacing the media every 2-3 days. On the day of the 
experiment, the microfluidic chambers were removed. During the 
experiment, media was replaced with physiological saline [135mM 
NaCl (Sigma-Aldrich), 3.5mM KCI (Sigma-Aldrich), 2mM CaCl2 
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(Sigma-Aldrich), 1.3 mM MgCl2 (BDH), 10mM Hepes (Ther- 
moFisher Scientific) and 20mM D-Glucose (Invitrogen)],”’ which 
was constantly replaced so that the osmolarity stayed within physio- 
logical conditions. 


B. Atomic force microscopy 


Atomic force microscopy experiments were conducted using 
either an MFP-3D-BIO AFM (Asylum Research) mounted on an 
Olympus IX-71 inverted optical microscope or a Bioscope-3 with 
Extender Module mounted on a Zeiss Axiovert s100tv. In both cases, 
the sample was mounted in a fluid cell, with access for the AFM can- 
tilever, and viewed from the bottom with the optical microscope at 
either 40x (air) or 100x (oil immersion) Zeiss objective lenses. For 
force measurements, the Nanosensors qp-SCONT cantilever was 
used with spring constant 0.01N/m (normal force measurements) 
or 0.09N/m (lateral force measurements), and partial gold coat- 
ing to minimize drift due to temperature changes and adsorption. 
Cantilevers were calibrated using the Sader method for normal and 
lateral spring constants. ~~ 


C. Pipette micromanipulation 


Cell media for 7-21 DIV neurons was replaced with 100nM 
of the live cell fluorogenic F-actin labeling probe (Si-R actin, 
Spirochrome) in 2mL of cell media. Immediately after, 10um beads 
coated for 24 hours with 100ug/ml PDL as described previously’ 
were added such that the probe and the beads were incubated 
with the neurons for 6-9 hours. The medium containing the probe 
was removed along with most unattached PDL-coated beads and 
replaced by physiological saline solution. The neurons were imaged 
using a Zeiss Axiovert 200M microscope and a 63x objective (Zeiss), 
with the F-actin probe illuminated by a xenon arc bulb (Sutter 
Instruments). Pipettes (King Precision Glass) of inner and outer 
diameters of 1mm and 1.5mm respectively were pulled using a Sut- 
ter Instruments P-87 pipette puller to a tapered opening of between 
2-6um. They were positioned using an Eppendorf InjectMan NI 2 
micromanipulator, and negative suction was applied via tubing con- 
nected to a syringe. Initially a positive pressure greater than the 
capillary pressure was applied to generate a positive flow to avoid 
collection of debris on the pipette. Once the pipette opening was 
manipulated to be in contact with a bead adhered to a bundle of 
axons, a negative pressure was applied to the pipette, picking up the 
bead and initiating a neurite (Figure 1). The bead was manipulated 
vertically ~3um to avoid scraping the neurite along the surface of the 
dish, then it was moved at a constant velocity of 0.1-0.5um/s parallel 
to the glass surface. The neurite was manipulated 100-250um, then 
deposited on another bundle of axons by applying positive pressure 
to the pipette. 


D. Axotomy and reconnection 


For AFM axotomy and reconnection, the Bruker MLCT-C can- 
tilever (spring constant 0.01N/m) was used to axotomize the axon, 
while the Bruker MLCT-D cantilever (spring constant 0.03N/m) 
with PDL coated bead on the same substrate was used to recon- 
nect the axotomized axon. For pipette axotomy and reconnection, 
pipettes with a flexible ending of spring constant 0.01N/m were 
pulled such that they could be pushed into the glass surface without 
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FIG. 1. (a) A pulled pipette (triangular shadow at top of image) applies suction to a 
PDL-coated bead (circled) and initiates a neurite by pulling upwards ~3um. (b) By 
pulling horizontally on the bead (circled) at a constant velocity of 0.1-0.5um/s, the 
neurite (indicated by the arrow) is extended. 


breaking. To avoid stretching the axons, the sample stage piezo was 
moved using a step function, moving the sample as fast as possible 
to make a clean cut. To test axonal fusion, a membrane impermeable 
dye (Alexa Fluor 488 Hydrazide) was injected at 10mM concentra- 
tion in an intracellular solution as previously described'® by using 
a patch pipette with a 2-6um opening and an Axon Instruments 
Axopatch 200A system. 


E. Beading the AFM probes 


1 drop of either polystyrene 10um beads (Polysciences) or Sil- 
ica 60pm beads (Microspheres-Nanospheres) was deposited on a 
microscope slide (Fisherbrand). The beads were dried and sepa- 
rated by using pressurized Nitrogen gas. Another adjacent slide was 
coated with E-30CL, a 2-part epoxy (Loctite). The tip was low- 
ered to coat the underside of the cantilever with glue, then moved 
over to the adjacent slide to adhere the bead to the glue. Con- 
stant observation using an inverted optical microscope makes this 
a routine process with a high yield. For lateral AFM measurements, 
a 60um and 10um bead were glued on top of each other such 
that the 10m bead contacted the axons. The 60pm bead served 
to increase the lever arm, reducing the cantilever torsional stiff- 
ness to 0.09N/m and to allow the measurement of the torsional 
deflection sensitivity. The torsional stiffness sensitivity was cali- 
brated by laterally deflecting the cantilever on a cleaved GaAs surface 
glued to the surface such that the contacted surface was 90° to the 
horizontal.” 


F. Data acquisition and analysis 


F-actin data was acquired using Northern Eclipse imaging soft- 
ware and a QImaging Retiga EXi camera, taking images every 4-30s. 
Exposure time was ls and gain 60% using a custom macro in the 
Full Control setting. The neurites were manually kept in focus for 
the duration of the experiment. The image stack was analyzed by 
drawing a line on top of the neurite and using the ImageJ Multi 
Kymograph plug-in v3.0.1. This produced a kymograph, which is 
constructed from one intensity line profile of a stack of images. 
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It is an image of position as a function of time, allowing the measure- 
ment of the speeds of F-actin polymers in the neurite as a function 
of time elapsed since pulling. These were analyzed by hand by trac- 
ing 15-30 actin trajectories per kymograph (each kymograph was 
generated from one neurite). All fits were done in MatLab using 
Isqcurvefit to obtain the variable values and nlparci to obtain an 
uncertainty. Data from multiple experiments was presented as mean 
+ SD. Each trajectory also gave a starting position and ending posi- 
tion which was converted to a starting position of actin in neurite 
and total movement of actin. AFM initiation curves were acquired 
by performing a force distance curve at 0.5um/s using IgorPro on the 
Asylum MFP3D AFM system. Data acquisition rate was 1kHz, with 
most of the approximately 16m dynamic range of the piezo being 
used. Subsequent pulls were done in the z-direction in the same 
way or by pulling parallel to the dish surface (in the y-direction), 
allowing pulls up to 50um. Pulls in the y-direction were performed 
using a function generator (Stanford Research Systems DS 345) by 
inputting a sawtooth signal into the piezo through the MFP3D Con- 
troller and pulling at 0.5um/s. The cantilever signal was acquired 
at 10kHz acquisition rate through MatLab which was connected 
to a National Instruments data acquisition card with 4096 pixel 
resolution. 


Ill. RESULTS AND DISCUSSION 
A. Building a neural network from neurons 


A simple artificial neural network for image recognition of the 
hand-written numbers 0-9 consists of 16x16 inputs,” 2 deep layers 
of 15 neurons each and 10 outputs.” The same configuration of neu- 
rons was then chosen as shown in Figure 2 to get an estimate of how 
long it would take to build a neuronal network from neurons. In a 
perceptron, every neuron from a previous layer is connected to every 
neuron from the next layer. This means that the total manipulation 
distance for the setup proposed would be given by: 


d=s V(n+1-i? + (j-k-1)? 45D 14 (j -k 
jk 


ijk 


1+(j-k-2) (1) 


Where the variable i represents the indices for the rows, j and k 
represent the indices for subsequent layers of columns in Figure 2 
and n is the total number of rows in the first layer. The first term 
in equation (1) corresponds to the total amount of neurite needed 
to be manipulated to connect every neuron from the layer of input 
neurons to the first deep layer of neurons. Similarly, the second and 
third terms in the equation correspond to the two remaining lay- 
ers being connected. The reason we sum over i, j and k in the first 
term, but only j and k in the second terms is that the first layer 
of input neurons has multiple rows (n rows) whereas the others 
only have one row. On a high-density multi-electrode array, sepa- 
rating the neurons by s=30 um would allow recording from each 
neuron. This means that the total manipulation distance for the req- 
uisite 4215 connections is 1.288 meters. In the following we will 
investigate if this is theoretically feasible using mechanical pulling of 
neurons. 

From a bio-engineering perspective, there is no reason to 
choose this task and neural network over another. However, the 


9, 075009-3 


AIP Advances 


16x16 
input 
neurons 


1x10 
output 
neurons 


Output = 0 bed 9 


FIG. 2. Multilayer Perceptron: Each neuron from the 16x16 block of input neurons 
will connect with each neuron from the first deep layer of 15 neurons. This is true 
for all adjacent layers (i.e. all neurons in one layer are connected to all neurons in 
the next layer). 


identification of hand-written digits has been extensively studied 
in the machine learning community, and deep learning techniques 
have been very successful at performing this task. This has led to it 
being called “the drosophila of machine learning” by Geoffrey Hin- 
ton,’ soit makes a simple, but good candidate for making an artificial 
neural network from a biological one. We do not intend on building 
this network ourselves but would simply like to show that it is pos- 
sible to build a neural network from neurons which can solve a real 
task. 
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B. Force requirements 


In order to wire a neural network, the manipulation method 
must be able to exert a large enough force to initiate and extend the 
neurites. Using an atomic force microscope to measure mechanical 
properties,” we found that the force required to initiate and extend a 
neurite varies significantly. To initiate a neurite, the maximum force 
was 1.1+0.7nN (Figure 3a) and to continue extending the neurite, 
the average force was 1.1+40.8nN (Figure 3b) for 5 different exper- 
iments at pulling speed 0.5um/s. The large variability in the force 
needed is likely due to the variation in number of neurites being 
pulled. For these experiments, we often initiated neurites from bun- 
dles of neurons, possibly initiating the pulling of several neurites. 
The values quoted above are thus the maximum forces needed to ini- 
tiate pulling of single neurites because the maximum force to initiate 
and pull one neurite will be less than or equal to the force obtained 
when pulling multiple neurites. 

During the initiation of a neurite, the force does not simply 
increase with stretch as in Hooke’s law. ° The force initially increases 
very quickly, followed by a decrease in force, then finally after 
much more extension, the force begins to increase gradually again. 
According to both Powers et al,” and Derenyi et al., the bead ini- 
tially pulls out a portion of the axon into a catenoid as shown in 
Figure 4. However, the boundary conditions of a catenoid are only 
stable when the neurite is below a specific length such that the neu- 
rite starts to collapse into a thin tube beyond this length and the 
force decreases.” As the pull continues, the tube starts to stretch, 
and the force increases again.’ Over the first 10um, ” this increase 
is approximately linear, but is clearly viscoelastic on longer length 
scales (Figure 3b). The simplest model that exhibits both viscous and 
elastic behavior is the Maxwell viscoelastic body and consists of a 
dashpot with viscosity 4 and a spring with stiffness constant k. The 
force F as a function of extension x for a neurite being pulled at a 
constant velocity v is then: 


F(x) = wo(1 - er) (2) 


The spring constant and viscosity obtained from fits as in Figure 3b 
from 5 different neurites is 56+35uN/m and 2.8+2.2mNs/m respec- 
tively. The significant variation in these values is again likely due to 
the variation of the number or diameter of neurites initiated. 


FIG. 3. (a) Initiation of Neurites from 5 
different experiments (maximum forces 
are F=1.98nN, 1.60nN, 0.81nN, 0.85nN, 
0.24nN). (b) Following initiation, the neu- 
rite is elongated, and the force is fit with 
a Maxwell viscoelastic model. 
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FIG. 4. The neurite is initially a catenoid until the boundary conditions become 
unstable, and it collapses into a tube. The tube stretches with elongation but 
maintains the same approximately cylindrical shape. 


C. Pulling speeds 


Using our micromanipulation technique, we can pull at a 
remarkably fast 0.5um/s to wire any two neurons together. Previ- 
ous work has been done to show that the neurite is functional after 
24 hours. The neurite contains actin, tubulin and neurofilament and 
it has been shown to be electrically connected to another neuron. 
As far as we can determine, these ‘pulled’ neurons are structurally 
and functionally not distinguishable from naturally grown ones. An 
intriguing question is why is it possible to pull so fast? Is the neu- 
rite growing at this speed or is the plasma membrane stretched and 
the neurite subsequently filled with cytoskeletal elements? These 
are biologically interesting questions that are important to under- 
stand the limits to how fast we can build a complex neuronal 
network. 

To answer this question, we fluorescently labelled F-actin, an 
important cytoskeletal element, and used it as a proxy for growth 
leading to biologically relevant structure and functionality. After 
stretching/extending the neurite hundreds of micrometers at 0.1- 
0.5um/s, the neurite is held at a constant length, and the maturation 
of the neurite can be observed. Immediately after stopping the exten- 
sion, actin appears to be pulled into the neurite. This can be seen in 
the kymograph in Figure 5a. 

As in section [II B, we can model the actin in the neurite as 
a Maxwell material, but this time with many Maxwell elements in 
series” (the exact details for the model are given in section III E). We 
thus derived the position of the i” element of F-actin cytoskeleton in 
the neurite as a function of time to be: 


di(t) = A(1-e"*) (3) 


Where A is the displacement of the actin and T is the characteristic 
time it takes for the actin to arrive to its final position in the neurite. 
We fit equation (3) to the data in Figure 5b, and we see that the char- 
acteristic time T is a constant independent of the actin position in the 
neurite (Figure 5c). Since T is a constant throughout the neurite, we 
can use the average T to determine the growth rate of the whole neu- 
rite. The average 7T for all the actin movement in a neurite, measured 
in 7 different experiments, is 15.5+9min. Using actin cytoskeleton 
as a proxy for growth, where the growth is assumed to be finished 
after t = 37 (when the distance the actin travelled is 95% of the total 
distance the actin will travel), these values translate to an effective 
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growth rate of 0.048+0.02um/s. This is calculated by dividing the 
total length of the neurite by 37 plus the pull time. The maximum 
growth rate achieved without neurite thinning and breaking for Fass 
et al. was 0.055um/s, indicating that the actual biological growth 
speed in our experiments is similar to the growth rates seen by oth- 
ers." In their experiments however, they were not able to pull faster 
without the neurite breaking. 

There are two main differences between the initiation of neu- 
rites in our experiments and those by others.’ The first is that 
we initiate the neurites de novo from axons whereas normally neu- 
rites are initiated de novo from the cell body. The second is that our 
axon forms a pre-synapse with the bead before initiation whereas 
we do not believe that is the case in other experiments. We believe 
that others initiate neurites using simple adhesion because they all 
initiate their neurites from the cell body, whereas pre-synapses are 
formed on axons. In addition to this, it takes at least 30 minutes to 
form a synapse with a PDL-coated bead.'° Bray left the polylysine 
coated pipette in contact only “10 minutes or so”, Heidemann’s 
group initiate neurites immediately after coming in contact with 
the cell," and the Integrin-coated beads added to the cells by 
Fass et al., “ were added once the neurons were in the manipula- 
tion setup, meaning they were not in contact long before the neu- 
rites were initiated (the exact time was not specified).'* Heidemann’s 
group also frequently pulls the growth cone of already formed axons, 
however, the fact that the axon is adhered to the surface slows the 
growth rate down because the adhesion increases its viscosity.” In 
our experiments, the neurite is always suspended in the solution, so 
we do not have this issue. It is possible that, in our experiments, the 
presence of an already formed axon adjacent to the neurite with a 
strong connection in the form of a synapse is better able to provide 
cytoskeleton and plasma membrane than a cell nucleus on its own, 
allowing for quicker pulling and the generation of a ‘guiding struc- 
ture’ which later fills with the cytoskeletal components present in all 
axons. 


D. Network robustness 


Once the neural network has been trained using images pro- 
vided openly in the NIST database,’ we would be able to deter- 
mine the contribution of each neuron and its connections to the 
success of the network. To test this experimentally, we could sever 
individual axons (axotomy) and then reconnect them. In Figure 6, 
we axotomized an axon using an atomic force microscope (AFM) 
cantilever, then used a PDL-coated bead glued to another AFM can- 
tilever to micromanipulate one axotomized end into contact with 
the other. Subsequent experiments were encouraging but incon- 
clusive as to whether the two axotomized ends fused. The newly 
grown neurites did show transport in both the proximal and dis- 
tal ends, but it is unclear whether there is transport across the 
reconnection. 

In further experiments designed to see if fusion between the two 
axotomized ends of the axon can occur simply by pressing the two 
ends together, we used patch clamp to input a membrane imper- 
meable dye into the nucleus following axotomy and reconnection. 
In one experiment, the dye was found in the distal end of the neu- 
rite. This appears to show fusion, but we only observed it once. 
There are a few reasons why fusion may not be observed: the axon 
forms a synapse with the bead (so wouldn’t fuse with the distal part 
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FIG. 5. Movement of actin in the neurite after extension: (a) Kymograph of the actin starting 9.5min after extending neurite. The bead is at the top of the image, and the 
neurite extends straight down as in the drawing on the left. (b) Traces of actin from the kymograph in (a) with equation (4) fit to the data. (c) The characteristic time z, a fit 
parameter in equation (4), is independent of the starting position of actin in the neurite. The characteristic time is equal to the spring constant divided by the viscosity of the 
neurite. (d) The total movement of actin has an inverse linear dependence on the starting position of actin in the neurite. 


of the axon), the cell would sometimes undergo Wallerian degen- 
eration following axotomy, finding the correct nucleus to patch to 
experimentally demonstrate successful fusion was difficult as there 
were many axons taking similar paths, and it is possible that the 
fusion does not occur every time. The fact that we did not observe 
routine fusion following axotomy is unsurprising due to the above 
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mentioned challenges. Furthermore, it has also not been observed 
in mammalian axons to our knowledge (except by using methods 
which break down the membrane structure).™ It has been observed 
to occur by axonal regeneration in C. elegans, crayfish, earthworm 
and leech.” Irrespective of whether the proximal and distal ends of 
the severed neuron can fuse using our micromanipulation technique 
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FIG. 6. Axotomy and Reconnection. (a) A single healthy axon is cut along the dash line using a sharp cantilever. (b) Axon following axotomy. (c) A PDL-coated bead glued 
to bottom of cantilever (large black object) forms a synapse with proximal end of the axotomized axon. (d) Force applied by cantilever and bead on axon generates a new 
neurite. (e) Neurite reconnects the axon’s proximal and distal ends. The axon appears to be reconnected and has transport in the proximal end, but not definitively in the 


distal end. 


and by pressing the two ends together, it has been routinely shown 
that the two ends can be fused by using a technique called micro- 
electrofusion.™>® This method works by dissolving the membrane 
using high electric pulses such that the membrane fuses into a sin- 
gle tube when it re-solidifies. We have showed here that by using 
microelectrofusion and our micromanipulation technique, we can 
systematically study the role of individual connections in the neural 
network. 


E. Model of the mechanical properties of the neurite 


In sections II B and III C, we used a Maxwell viscoelastic model 
(Figure 7b) to fit force data and actin movement data respectively. 
In the constant velocity extension curves in Figure 3b, the force 
increases, indicating stretch. This provides the motivation for the 
elastic component in the Maxwell model. The viscous component 
allows viscous flow to relax the spring, shown by the force plateau in 
Figure 3b. Solving for the force as a function of extension at constant 
speed gives equation (2) which was used to fit the data in Figure 3b. 
Since the neurite is well modeled as a spring and dashpot in series, 
it seems reasonable to model the transport of actin along the length 
of the neurite similarly. One spring and dashpot in series is mathe- 
matically equivalent to many connected smaller Maxwell elements. 
Since we are examining the actin movement throughout the length 
of the neurite, we propose a viscoelastic model consisting of many 
Maxwell elements in series and derive the displacement of each 
element as a function of time (the model in Figure 7a shows only 
the springs to make the visualization of each spring’s displacement 
easier to follow). 


AIP Advances 9, 075009 (2019); doi: 10.1063/1.5086873 
© Author(s) 2019 


The flow of actin into the neurite occurs after the neurite is 
extended and the springs are stretched out (Figure 7a). The ten- 
sion from the stretching pulls more material into the neurite by 
overcoming the viscous resistance, relieving tension and effectively 
elongating the neurite. Each spring in the neurite will feel the same 
force and will be stretched equally because the springs are arranged 
in series. In Figures 7a and 5c, we see that the total displacement d; 
of the proximal springs (those closest to the axon-neurite junction) 
will be much larger than the displacement of the distal springs (those 
closest to the bead). This is because the full displacement of the i™” 


b Maxwell Body 
| 
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FIG. 7. Physical model of the neurite during extension: (a) When modelling the 
neurite as many springs (and dashpots, not shown) in series, we can predict where 
each spring will be relative to each other in the neurite by calculating d; as a 
function of d;—4. (b) Maxwell viscoelastic body: the movement of the actin in time 
cannot be instantaneous because there is viscous flow. We model this by putting 
a dashpot in series with each spring in the neurite. 
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spring is equal to the total amount of stretch of all the springs in 
front of it: 


Fe ax ixi(t) (4) 


Here, N is the total number of springs in the neurite, x is the total 
amount of initial stretch in the neurite, and x; is the instantaneous 
stretch in each spring. Each elastic spring element is accompanied by 
a viscous element (modelled by a dashpot) to represent the resistance 
of the neurite to flow. Each spring and dashpot will experience the 
same amount of force, so that we can write the force applied on each 
element in the neurite as: 


Fi = ksxj = Hadi = di) (5) 


We have assumed that the spring constant ks and the viscosity wg 
of all constituents are the same, which agrees with the linear trend 
in Figure 5d. Combining equations (4) and (5), and solving the 
resultant differential equation gives: 


di(t) = a (1 -e') (6) 


Which describes the displacement of the i” element of the neu- 
rite as a function of time. Equation (3) is the same as equation (3), 
only we have now defined A = ix/N and T = a/ks. As noted in sec- 
tion III C, the characteristic time T = q/k, is a constant independent 
of the actin position in the neurite (Figure 5c). This indicates that 
the tension is distributed equally throughout the neurite, allowing 
all constituents to relax together, and providing evidence there is 
a similar composition along the length of the neurite. The neurite 
displays a remarkable ability to stretch itself, with the actin pulled 
mechanically into the neurite. 

Equation (6) also indicates that the total movement of actin in 
the neurite should be smaller for actin that starts distally and that 
this should have a linear dependence. This is what we observe in 
Figure 5d. The deviation of this line from the y = —x line is the total 
length of springs in the neurite measured in equilibrium, L = NI, 
where lis the length of one unstretched spring. In terms of variables 
defined in the model, Figure 5d is a plot of ix; + L vs (i/N)x. As we 
would expect, neurites which were pulled a longer distance had both 
a longer length L and initial stretch x (data not shown). 

The model describes all aspects of the data, providing evidence 
that the flow of actin is mechanically driven by the force applied 
to the neurite. The neurite does not grow at the pulling rate set by 
the user, but instead stretches, and later fills in with a characteristic 
time which depends on the spring constant and the viscous resis- 
tance. When compared to other mechanically induced growth in 
neurites, the effective growth rate of these neurites is similar. How- 
ever, because we are able to pull much faster, this allows the pos- 
sibility to wire a complex neural network in a comparatively small 
amount of time. 


F. Optimal manipulation 


With our micromanipulation technique, we can pull at 
0.5um/s, meaning it would take 30 days of pulling time to wire the 
proposed multilayer perceptron network’s 1.288 meters total length. 
This is prohibitively slow: The cells will die well before the 30 days 
of manipulations are complete, as the manipulation is not performed 
in an incubator with optimal conditions for neuron survival. A more 
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realistic maximum time for completing these manipulations before 
the cells die is empirically about 24 hours; an effective speed up of 
about 30 times is thus necessary. The only solution to reduce the 
manipulation time by this much would be to pull multiple beads 
simultaneously. 

The necessary criteria for wiring an arbitrary neural network 
are: 


1. The setup must keep cells alive for many hours. 
. It must be possible to manipulate in all 3 dimensions. 
3. Most importantly, as described in section III B, the method 
must be able to exert a large enough force to initiate and extend 
the neurites. 


There are many techniques for manipulating microspheres in a 
dish. Optical tweezers can be multiplexed, but the trap force maxi- 
mum is generally about 100pN and the laser can cause photodam- 
age and heat the sample significantly, both leading to cell death.” 
Acoustic force spectroscopy and centrifugal force spectroscopy both 
lack multidimensional motion.” An atomic force microscope can- 
tilever array has many positives, but it cannot effectively release the 
beads as these are usually glued to the end of the cantilevers.” By 
making a hole in the bottom of the cantilevers, it could be possible to 
use suction to pick up the beads, but this method still lacks indepen- 
dent control over each bead.'' Magnetic traps have been multiplexed 
by positioning a magnet far away from the beads, such that all beads 
feel the same field. However, the multiplexing abilities mean that the 
magnets are positioned quite far away such that the maximum force 
is 260pN, and in most systems significantly less.“ The multiplexed 
traps also lack independent control as they exert the same force on 
all beads. 

The best option seems to be to use 2-3 magnetic pole pieces 
similarly to Fass et al.,'°. Since the pole pieces are mounted on a 
micromanipulator, each pole piece would give control over another 
dimension, the pole pieces can be brought into proximity with the 
magnetic bead of choice (allowing a large range of force control and 
bead selectivity) and this method does not kill the cells. Using an 
electromagnet, the force dependence on distance is a power law’ 
which allows manipulating specific beads based on proximity to the 
pole pieces. Also, the force depends linearly on the current flow- 
ing through the solenoid” allowing quick adjustments to the force 
applied. Using a 4.5m bead, and by varying the distance and the 
current, it is possible to apply forces that range from 15pN up to 
many nanonewtons. “ 

This method could be multiplexed by picking up each bead in 
a column (see Figure 2) in one pass, such that many beads will all 
be pulled together. An appropriate field gradient could be one where 
a bead 14um away would feel a force of around InN, sufficient to 
initiate and extend a neurite. Using this field, a bead on an adjacent 
axon another 30um away (and thus a total of 444m away), would 
feel only 0.1nN force, so it would not be initiated. For reference, a 
bead 10, 20, 30, 40 and 50um away from the pole piece would feel a 
force of 2.0, 0.5, 0.22, 0.12 and 0.078nN respectively. In the situation 
where a bead on an adjacent axon was for example only 20um away, 
it could be inadvertently initiated. This should be avoidable though 
because each bead is placed on the axon by the user and the multi- 
electrode arrays are separated by 30m (although some optimization 
might be necessary here). Using these parameters, many beads could 
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be picked up sequentially and connected to their common destina- 
tions simultaneously. Each pass would thus connect one column of 
16 neurons in the first layer to every neuron in the entire row of 15 
neurons in the next layer. 

Since the input neurons form a 16x16 square of neurons, each 
column of this square could be pulled at one time simply by pick- 
ing up a bead and moving on to the next row of neurons to pick up 
the next bead and initiate another neurite. To connect these newly 
initiated neurites to the dendrites of the next layer’s neurons, the 
neurites could be connected by wedging them under a non-magnetic 
bead sitting on the target neuron’s dendrites. In this way, the col- 
umn of neurons will have their neurites connected to each neuron’s 
dendrites in the next layer’s row in one single pass. This would 
reduce the amount of pulling time from about 42 days in sequen- 
tial pulling, as calculated at the beginning of this section, to about 
10.3 hours. Of course, for this to work, 296 beads would have to 
be precisely placed on the axons and dendrites throughout the dish. 
Placing a bead takes about 1 minute, so this would add about 5 more 
hours. Adding another hour for the user to operate the setup and 
make decisions in real time implies that it should be possible to wire 
the entire circuit in only 17 hours. Based on these considerations, 
using multiple magnetic pole pieces to connect this circuit seems 
feasible. 


G. A learning biological neural network 


Once the neural network has been successfully wired, the con- 
nections will be initially firing randomly, and will not be able to 
process a hand-written digit. In an artificial neural network, teach- 
ing a neural network to recognize a digit is done through a method 
called backpropagation. Each input neuron receives a pixel value 
from 0 to 1 from the initial 16x16 image, and the signal is passed 
through the layers to the output layer. As the signal passes from 
one layer to the next, the activation energy (neuron initial value), 
the weight (factor multiplied with the activation energy) and the 
bias (offset added at the end) will be scaled to determine what 
value between 0 and 1 the next neuron will take.’ Teaching the 
network adjusts those three variables for every connection in the 
network appropriately to maximize the chance of outputting the 
correct number. Backpropagation works by calculating the cost 
function (the sum of the difference between the output and the 
input), and figures out what values of activation energies, weights 
and biases will give the correct value. This is done by minimiz- 
ing the gradient of the cost function on a large set of training 
data. 

In a biological neural network, Hebbian theory says that neu- 
rons can adjust the strength of their connections by increasing the 
number of synaptic connections or by changing the nature of their 
synapse (e.g. by increasing synaptic vesicle exocytosis) in a process 
called long-term potentiation. "° Long-term potentiation is analo- 
gous to increasing the weight of a connection in an artificial neu- 
ral network. Pre- and post-synaptic neurons depolarizing together 
during a high frequency stimulation can cause long-term potentia- 
tion.“ Certain pathways may be inadvertently strengthened, but with 
the multi-electrode array, those pathways can be discouraged by 
hyperpolarizing select cells, preventing them from being strength- 
ened. Bienenstock, Cooper and Munro (BCM) developed a theory 
which builds on Hebbian theory and says that when a pre-synaptic 
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neuron is stimulated, but the post-synaptic activity is below a certain 
threshold, long-term depression will occur.” Long-term depression 
is the functional opposite of long-term potentiation and is analogous 
to decreasing the weight of specific connections. By inducing either 
long-term potentiation or depression, specific connections can be 
either encouraged or discouraged.**”” Synaptic plasticity can be 
induced using other methods such as: the pairing of depotentia- 
tion in pre- and post-synaptic neurons, » using naturally occurring 
firing patterns to induce LTP,” or by using N-methyl-D-aspartate 
to induce LDP.” Non-synaptic forms of plasticity which affect all 
the synapses in a cell” or membrane geometric properties’ also 
exist. 

A significant difference between a neuron and a computer 
is that a neuron will only fire if it reaches a threshold polariza- 
tion, and its strength of polarization is always the same, which 
means that the analog of activation energy in an artificial neural 
network is not the same as changing the strength of the action 
potential in a biological neural network. Instead, information is 
encoded through stimulation pulses. We could make high fre- 
quency pulses in a biological neural network analogous to higher 
activation energies in an artificial neural network. The pixel val- 
ues that were encoded in the artificial neural network with a value 
between 0 and 1 could instead be encoded using higher and lower 
frequency inputs. Since the multi-electrode array allows both the 
stimulation and recording of action potentials of all the neurons 
in the dish, the entire network could be analyzed with the gra- 
dient of the cost function as in an artificial neural network. By 
systematically changing the long-term potentiation and depression 
of specific connections, the network could then be trained using 
backpropagation. 

Assuming an artificial neural network with the exact topogra- 
phy described in section III A is successfully built, there will still 
be a number of challenges in the actual training of the neural net- 
work and the interpretation of its outputs. These are challenges 
present in any artificial neural network, but with the added com- 
plexity that the network is slower to train, the activation energy here 
is encoded as a frequency and the neuron may compute information 
in a fundamentally different way. 

The biggest challenge would be if the biological neural net- 
work is unable to learn at all and the error rate on the training 
dataset never decreases significantly (underfitting). The strength of 
the connections between neurons are altered by inducing physiolog- 
ical changes to each connection, meaning that the neurons will take 
much longer to train than an artificial neural network. This means 
that it may not be possible to do more than 1 epoch (i.e. one pass 
through the entire training dataset, which for the MNIST dataset is 
60000 hand-written digits), and hence the data will be underfit.’ To 
compensate for this limitation, it may be more efficient to increase 
the number of training examples between each change to connection 
strengths. Another reason the network is underfitting could be due 
to fundamentally different computational methods between an arti- 
ficial neural network and a biological one (e.g. because the frequency 
of the signal may not encode the activation energies properly). We 
know there are differences in how the two networks compute, and 
this may necessitate a different approach for training the network. 
Rather than specifying the exact algorithm for computing to the net- 
work, letting it figure out how to best process the information itself 
might give insights into how a biological neural network naturally 
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computes. However, this will be difficult as we do not yet have a 
clear idea of how the network computes, so it will be difficult to 
know how to train the network (i.e. what behavior to encourage or 
discourage). 

The opposite scenario where the network performs well on 
training data but does not perform well on test data is called overfit- 
ting. In this case, the capacity of the network will need to be reduced. 
In an artificial neural network, one way this is done is by using dif- 
ferent types of non-linear functions for calculating the activation 
energies. However, for a biological neural network, the neuron fires 
if depolarized below a certain threshold and this is intrinsic to the 
neuron, so it is not possible to change this. Instead, other techniques 
for reducing the capacity of the network should be employed, such 
as reducing the number of neurons in the deep layers or reducing 
the number of epochs. Following these adjustments, if the network 
can successfully classify the hand-written digits from the test dataset 
(10000 digits), then we can be satisfied that the network has truly 
learned how to perform a complex task. We believe that the pro- 
cess of getting to this point will be important in understanding the 
difference between the biological neural network and the artificial 
one. 


IV. CONCLUSION 


We believe that building an artificial neural network from bio- 
logical neurons will help to understand what is fundamentally differ- 
ent between a brain and a machine. We propose a method to train 
the network by using backpropagation, which is used on artificial 
neural networks, by systematically changing the weight of the con- 
nections between neurons and by using frequencies to encode what 
would normally be entered as the activation energies of an artifi- 
cial neural network. We propose using a multi-electrode array, to 
record all action potentials and stimulate any neuron, which would 
allow us to measure and systematically change any of the weights 
of the connections between neurons, capabilities which are not cur- 
rently available for any complex network. The bioengineering feat 
of wiring all the connections into a complex biological neural can 
be done by using our micromanipulation technique which can pull 
300 times faster than average natural growth. We have shown that 
the maximum forces required to initiate and elongate these neurites 
is within the capabilities of a magnetic trap. We have developed a 
model which describes the remarkable mechanically induced growth 
in the neurite. Furthermore, we have shown that it could be possi- 
ble to test the robustness of this network by axotomizing specific 
axons and then reconnecting them. We believe that all the neces- 
sary knowledge and technology is available to build and teach an 
artificial neural network from biological neurons which will allow 
new insights into computation and learning in both machines and 
in brains. 
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